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ABSTRACT 

Presented  in  this  paper  is  an  on-going  project  to  develop 
a  new  generation  of  virtual  human  models  that  are  highly 
realistic  in  terms  of  appearance,  movement,  and 
feedback  (evaluation  of  the  human  body  during  task 
execution).  Santos™  is  an  avatar  that  exhibits  extensive 
modeling  and  simulation  capabilities.  It  is  an 
anatomically  correct  human  model  with  more  than  100 
degrees  of  freedom.  Santos™  resides  in  a  virtual 
environment  and  can  conduct  human-factors  analysis. 
This  analysis  entails,  among  other  things,  posture 
prediction,  motion  prediction,  gait  analysis,  reach 
envelope  analysis,  and  ergonomics  studies.  There  are 
essentially  three  stages  to  developing  virtual  humans: 
(1)  basic  human  modeling  (representing  how  a  human 
functions  independently),  (2)  input  functionality 
(awareness  and  analysis  of  the  human’s  environment), 
and  (3)  intelligent  reaction  to  input  (memory,  reasoning, 
etc.).  This  paper  addresses  the  first  stage.  Specifically, 
we  discuss  a  new  human  model  in  terms  of  mechanics 
and  appearance.  We  present  an  optimization-based 
approach  to  kinematic  and  dynamic  motion  analysis. 
This  approach  allows  the  avatar  to  operate  with 
complete  autonomy  rather  than  with  dependence  on 
stored  animations  and  data,  or  restrictions  associated 
with  inverse  kinematics.  With  dynamic  analysis,  it  is  not 
necessary  to  solve  equations  of  motion.  A  novel 
approach  for  determining  reach  envelopes  is  also 
presented,  and  this  approach  provides  a  unique  tool  for 
ergonomic  studies.  Methods  for  evaluating  the 
physiological  status  of  the  virtual  human  as  tasks  are 
completed  are  discussed.  Finally,  additional  on-going 
research  is  summarized.  The  result  is  an  exciting  step 
towards  a  virtual  human  that  is  more  extensive  and  more 
complete  than  any  other. 

Keywords:  Virtual  humans,  optimization-based  model, 
human  performance  measures,  real-time  reaction. 


INTRODUCTION 

The  application  of  virtual  humans  in  industry  is 
expanding,  especially  with  respect  to  vehicle  design  and 
manufacturing.  In  fact,  potential  applications  span  a 
variety  of  areas  that  include  the  following:  crash 
simulation  (vehicular  occupant  injury),  slip/falls 
(environmental  injury),  orthopedics  (joint  and  tissue 
mechanics),  gait  simulation  (rehabilitation),  task 
simulation  (virtual  factory),  and  sports-performance 
simulation.  The  primary  benefit  of  virtual  humans  in 
these  areas  is  that  they  can  be  incorporated  directly  into 
CAD  drawings.  Then,  engineers  can  perform  extensive 
analysis  concerning  ergonomics  and  design,  during  the 
early  stages  of  the  product  development  process.  This 
alleviates  the  need  for  physical  mock-ups  and  thus 
shortens  product  design  cycle  times  and  lowers 
development  costs. 

Work  on  this  subject  spans  at  least  two  decades. 
Cyberman  (Blackeley  1 980)  is  an  example  of  one  of  the 
earliest  virtual  humans.  Chrysler  Corporation  developed 
it  for  the  automobile  industry,  in  order  to  define  and 
analyze  acceptable  limb  and  body  locations  for  a  human 
model  within  a  virtual  environment.  Specifically,  it  is  used 
to  analyze  virtual  drivers  and  passengers,  and  their 
activities  in  and  around  a  car.  However,  Cyberman  is 
only  a  simple  wireframe  model.  Additional  virtual 
humans  of  similar  complexity  have  also  been  developed. 
Combiman  (Bapu  et  al.  1980)  was  designed  at  the 
Aerospace  Medical  Research  Laboratory  in  order  to 
determine  human  reach  capacity,  and  it  has  been  used 
for  aircraft  cockpit-configuration  design  and  evaluation. 
Sammie  was  designed  at  the  University  of  Nottingham 
for  general  anthropometric  analysis  and  design  (Kingsley 
et  al.  1981).  Additional  anthropometric  modeling 
programs  have  been  developed  at  the  Boeing 
Corporation  (Boeman,  Car- Harris  et  al.,  1980),  at 
Rockwell  International  (Buford-  Fetter,  1982),  and  at  the 
University  of  Pennsylvania  (Bubbleman- Badler  et  al., 


1979).  Because  the  appearance  of  initial  virtual  humans 
has  been  far  from  realistic,  considerable  research  has 
been  conducted  in  an  effort  to  improve  realism.  For 
instance,  Badler  et  al.  (1993)  and  Thalmanns  (1990, 
1993,  1996)  develop  models  based  on  the  combination 
of  multiple  cylinders.  In  addition  to  effort  focused 
towards  visual  appearance,  work  has  been  conducted 
with  autonomous  perception,  intelligence,  and  behavior 
(Thalmann  et  al.  1995;  Emering  et  al.  1997;  Noser  and 
Thalmann  1994,  1995,  1996;  Noser  1997). 


Recently,  several  companies  have  developed  and 
marketed  relatively  advanced  virtual-human  products. 
Badler  et  al.  (1999)  created  Jack™  software,  which  is  a 
scalable  human  model  with  a  flexible  spine,  and  limbs 
that  can  be  articulated  through  inverse  kinematics. 
Jack™  can  be  positioned  for  reach  analysis  and  visual 
interference  analysis  as  well  as  strength,  posture,  and 
motion  analysis.  However,  Jack™  cannot  move 
independently,  so  the  user  must  manipulate  the  various 
limbs  manually.  Safework™  was  developed  at  the  Ecole 
Politechnique  (Montreal).  It  involves  six  basic 
mannequins  that  work  in  conjunction  with  an 
anthropometries  module  to  create  human  beings  from 
any  population.  Safework™  can  be  used  for  extensive 
ergonomic  analysis.  It  incorporates  inverse  kinematics 
with  a  full  human  model,  which  integrates  with  CATIA 
software.  TechMath’s  Ramsis™  and  Anthropos™  are 
human  modeling  systems  for  ergonomic  analysis, 
design,  and  visualization,  which  are  also  manipulated  by 
a  user.  Ramsis™  is  used  extensively  for  designing 
automobile  interiors  and  airplane  cockpits.  Anthropos™ 
is  a  plug-in  for  3D  Studio  with  excellent  visualization, 
ergonomic  analysis. 


Although  the  different  systems  mentioned  above  are 
useful,  all  of  them  are  based  on  predetermined  empirical 
data.  Alternatively,  our  approach  governs  motion  based 
on  human  performance  measures  that  act  as  objective 
functions  in  an  optimization  problem.  The  above- 
mentioned  systems  require  subjective  manipulation  and 
cannot  quantify  human  performance  measures,  as  the 
optimization-based  approach  can. 


In  this  paper,  we  present  newly  developed  facets  of  an 
optimization-based  avatar  that  promises  to  be  the  next 
generation  of  virtual  humans.  This  avatar  is  called 
Santos™.  The  work  presented  in  this  paper  is  part  of  an 
on-going  project  to  develop  a  system  with  which  an 
avatar’s  motion  is  governed  by  a  variety  of  human 
performance  measures  that  are  incorporated  in  a  novel 
optimization-based  approach  to  posture  and  motion 
prediction.  First,  we  introduce  the  human  model, 
discussing  the  mechanics  of  skeletal  motion  and  the 
details  of  its  realistic  appearance.  Then,  several  key 
functionalities  are  presented.  We  also  discuss  the  ability 
to  receive  feedback  concerning  the  avatar’s  physiology. 
Finally,  on-going  research  is  reviewed  briefly. 


HUMAN  MODEL 

SKELETAL  MODEL 

In  this  section,  we  give  a  brief  outline  of  the  mathamtical 
foundation  for  the  human  skeletal  model  used  to  control 
the  motion  of  Santos™.  Essentially,  this  model  relates  a 
set  of  joint  angles  to  the  Cartesian  coordinates  of  point 
on  the  avatar.  A  human  skeleton  can  be  modeled  as  a 
kinematic  system,  a  series  of  links  with  each  pair  of  links 
connected  by  single  degree-of-freedom  (DOF)  joints  as 
shown  in  Fig.  1 .  Therefore,  a  complete  human  body  can 
be  modeled  as  several  open-loop  chains,  one  of  which  is 
shown  in  Fig.  1 . 

In  Fig.  1 ,  qt  is  a  generalized  coordinate  and  represents 
the  rotation  of  a  single  basic  revolute  joint. 


Fig.  1  Definition  of  a  kinematic  pair  (e.g.,  a  revolute  joint) 


Each  joint  in  the  skeletal  model  is  represented  by  one, 
two,  or  three  revolute  joints.  q  =  [ql,...,qnf  &R"  is  an  n- 

dimensional  vector  of  generalized  coordinates,  where  n 
is  the  number  of  DOFs.  The  end-effector  is  a  point  on 
the  virtual  human,  typically  the  tip  of  the  index  finger. 
x(q)  represents  the  position  of  the  end-effector  in 

Cartesian  space  and  is  a  function  of  the  joint  angles. 
Using  the  Denavit-Hartenberg  (DH)  method  (Denavit  and 
Hartenberg,  1955),  it  is  determined  using  a  series  of 
transformation  matrices  as  follows: 

-ii'n '  'kj'  p,  o) 

'■=i  ;-i 


x(q): 


*(q) 
y(  q) 
z(q) 


where  \  =  [x  y  zf.  'py  and  mr;  are  defined  as 
follows: 


'  R 


cos  qi  -  cos  ai  sin  qt  sin  at  sin  qt 
sin  qt  cos  ai  cos  qt  -  sin  at  cos  qt 
0  sin  at  cos  at 


(2) 


('_1)P,  =  [a,  cos  4,  a,  sin  qt  cl,  f  (3) 

q,  is  the  joint  angle  from  x;_,  axis  to  the  xi  axis,  di  is 
the  shortest  distance  between  x(  and  x(  axes,  at  is 
the  offset  distance  between  z,  and  z;_,  axes,  and  a,  is 


the  offset  angle  from  z,._,  and  z(  axes.  Details  of  this 
approach  are  discussed  by  Marler  (2004). 

APPEARANCE 

The  basic  human  model  not  only  involves  the  skeletal 
mechanics,  which  are  discussed  above,  but  also  the 
overall  appearance,  which  is  discussed  in  this  section. 
The  actual  three-dimensional  model  is  comprised  of  a 
“skin”  laid  over  the  skeleton.  This  “skin”  can  be  thought 
of  as  an  infinitely  thin  but  hollow  shell  that  defines  the 
avatar’s  shape  (Figs.  2  and  3). 


Fig.  2  Wire  frame  view  Fig.  3  Solid  view 

Once  the  solid  shape  of  the  avatar  is  created,  shaders 
and  textures  are  used  to  provide  the  avatar’s  shape  with 
the  visual  queues  necessary  to  create  the  illusion  of 
actual  human  skin,  as  shown  in  Fig.  4.  Technically,  a 
shader  is  a  compilation  of  effects  that  dictate  how  a 
three-dimensional  surface  responds  to  light.  Textures 
are  two-dimensional  images  that  are  projected  onto  or 
wrapped  around  three-dimensional  surfaces. 


Fig.  4  Textured  view  of  a  virtual  human 

By  themselves,  three-dimensional  models,  shaders,  and 
textures  can  provide  convincing  renderings  that  suggest 
human  form,  but  this  form  cannot  be  manipulated  until  it 
is  bound  to  a  hierarchical  joint  structure  representing 
Santos’™  skeleton  (Fig.  5). 


Once  defined,  the  avatar’s  skin  can  be  bound  to  the 
skeleton,  allowing  the  avatar  to  be  moved  in  a  human 
way.  Motion  for  the  joints  shown  in  Fig.  5  is  determined 
using  the  method  described  in  the  next  section  and  the 
mathematical  model  described  above.  However,  to 
visually  simulate  the  elasticity  of  real  human  skin  as  the 
joints  are  exercised,  the  motion  of  different  regions  of 
skin  must  be  related  to  the  motion  of  the  skeleton.  This 
is  done  using  a  well-known  static  animation  technique 
called  skin  weighting  and  addresses  the  aesthetic  issue 
that  would  otherwise  cause  a  three-dimensional  model 
to  tear  or  break  at  the  joints  when  joints  are  rotated,  as 
shown  in  Fig.  6. 


Fig.  6  Skin  with  unadjusted  weight 

Typically,  this  technique  is  accomplished  subjectively 
through  interactive  tools  -  much  like  using  a  can  of  spray 
paint  -  that  allow  8-bit  gray  level  values  to  be  applied  to 
joint-specific  regions  of  the  skin.  The  higher  the  gray 
level  value,  the  greater  the  effect  a  given  joint  has  on 
that  region  (Fig.  7). 


When  the  outer  appearance  of  the  avatar  is  coupled  with 
the  mathematic  model  (kinematic  system  of  joints),  the 
result  is  a  complete  human  model,  as  shown  in  Fig.  8. 


Fig.  7  Skin  with  adjusted  weight 


target  point.  The  final  optimization  formulation  is  given 
as  follows: 


Find:  q  e  Rn  (4) 

to  minimize:  Human  performance  measure(s) 
subject  to:  [x(q )  _  ]2  <  ^ 

q[  < q,  <  q f ;  i  =  l,2,...,n 

Other  components  of  the  formulation  are  discussed  in 
the  following  sections.  The  problem  in  (4)  is  solved 
numerically  using  the  software  SNOPT  (Gill  et  al,  2002), 
which  uses  sequential  quadratic  programming  (Arora, 
2004).  Details  concerning  the  different  components  of 
the  optimization  formulation  are  discussed  as  follows. 

Design  Variables  and  Constraints 

The  design  variables  are  the  generalized  coordinates  qt , 

which  represent  the  components  of  the  vector  q.  These 
design  variables  have  units  of  radians.  The  first 
constraint  in  (4)  is  the  distance-constraint,  and  it  requires 
that  the  end-effector  contact  a  predetermined  target 
point  in  Cartesian  space,  where  s  is  a  small  positive 
number  that  approximates  zero.  The  distance  between 
the  target  point  and  the  index  finger  should  be 
approximately  zero.  Technically,  this  constraint 
represents  distance-squared.  In  addition,  each  joint 
angle  is  constrained  to  lie  between  lower  and  upper 
limits,  represented  by  qf  and  q\!  respectively.  These 
limits  ensure  that  the  avatar  does  not  assume  an 
unrealistic  position  in  an  effort  to  contact  the  target.  They 
may  be  changed  easily  according  to  different  sets  of 
anthropologic  data. 


Fig.  8  Santos™,  a  complete  virtual  human 

POSTURE  PREDICTION 

In  this  section,  we  discuss  optimization-based  posture 
prediction,  which  provides  a  cornerstone  for  motion 
prediction.  An  overview  of  the  optimization  formulation  is 
presented  here,  while  details  concerning  the 
development  of  the  formulation  are  provided  by  Farrell 
and  Marler  (2004).  In  addition,  the  newly  developed 
ability  to  control  the  orientation  of  body  parts  is 
summarized. 

OPTIMIZATION  PROBLEM  FORMULATION 

In  general,  the  optimization  problem  entails  finding  the 
vector  of  joint  angles,  q,  in  order  to  minimize  one  or 
more  human  performance  measures  that  serve  as  the 
objective  functions  and  govern  the  motion  of  the  avatar. 
The  problem  is  constrained  by  limits  on  the  joint  angles 
and  by  the  requirement  that  the  avatar  must  contact  the 


Note  that  the  design  variables  are  continuous  and  can 
represent  infinitely  many  postures  given  a  specific  set  of 
constraints  (and  consequent  feasible  space).  This 
inherent  property  of  the  optimization  problem  provides  a 
significant  advantage  over  other  approaches  to  human 
modeling.  The  constraints  and  consequent  set  of 
potential  postures  are  essentially  dictated  by  the  avatar’s 
environment  and  by  the  human  model  that  the  avatar 
represents. 

Objective  Functions 

Various  human  performance  measures  provide  the 
objective  functions  for  the  formulation  in  (4).  Details 
concerning  these  performance  measures  are  provided 
by  Yang  et  al.  (2004b).  Here  we  present  an  overview. 
The  most  fundamental  of  these  functions  is  joint 
displacement,  which  is  given  as  follows: 

/joi„.D,sP  (q)  =  X wi  (qt  -  q?  f  (5) 

7  =  1 


qf  is  the  neutral  position  of  a  joint,  and  q N  represents 
the  overall  neutral  posture.  The  neutral  posture  is 
selected  as  a  relatively  comfortable  posture,  typically  a 
standing  position  with  arms  at  one’s  sides.  A  weight  wi 
is  introduced  to  stress  the  importance  of  a  particular 
joint.  Currently,  these  weights  are  determined  by  trial 
and  error.  Generally,  with  joint  displacement,  the  avatar 
gravitates  towards  the  neutral  position. 

Effort  is  similar  to  joint  displacement,  but  gfina/  replaces 
qf  and  represents  the  avatar’s  initial  position  or  starting 

position.  Thus,  when  effort  is  used  as  the  objective 
function  in  (4),  the  avatar  gravitates  to  its  starting 
configuration,  no  matter  what  that  configuration  may  be. 
This  performance  measure  is  most  significant  when  a 
series  of  target  points  are  selected,  with  the  posture 
changing  from  point  to  point. 

Using  delta-potential-energy  as  an  objective  function 
provides  an  alternative  approach  for  determining  the 
weights  in  (5).  Various  segments  of  the  avatar’s  body 
are  treated  as  lumped-masses.  The  total  change  in  the 
potential  energy  for  the  masses  is  minimized.  When 
determining  the  change  in  potential  energy,  the  initial 
configuration  is  always  set  as  the  neutral  position 
described  above.  In  this  way,  the  masses  of  the  different 
sections  of  the  body  essentially  provide  inherent  weights 
for  the  motion  of  the  different  segments. 

Finally,  discomfort  is  modeled  as  another  variation  on 
joint  displacement.  Again,  the  avatar  moves  towards  the 
neutral  position.  However,  this  function  incorporates 
three  facets  of  comfort:  1 )  the  tendency  to  move  towards 
a  generally  comfortable  position,  2)  the  tendency  to 
avoid  postures  with  which  joint  angles  are  pushed  to 
their  limits,  and  3)  the  idea  that  people  strive  to  reach  or 
contact  a  point  using  one  set  of  body  parts  at  a  time. 
Typically,  in  terms  of  upper  body  motion,  one  first  tries  to 
reach  a  point  using  one’s  arm.  If  that  is  unsuccessful, 
only  then  does  one  bend  the  torso.  Finally,  if  necessary, 
the  clavicle  is  extended.  Note  that  the  intent  in 
developing  this  performance  measure  is  not  necessarily 
to  quantify  discomfort.  Rather,  the  function  is  designed 
to  be  proportional  to  discomfort.  Only  its  relative  values 
(from  one  posture  to  another),  not  its  absolute  values, 
are  significant. 

ORIENTATION 

Often,  the  primary  concern  with  human  modeling  is  the 
analysis  of  gross  body  motion  and  posture,  and  the  only 
stipulation  when  trying  to  duplicate  human  motion 
concerns  the  position  of  a  body  part,  typically  a  fingertip. 
However,  when  a  model  is  required  to  interact  with  its 
surroundings,  then  the  orientation  of  different  body  parts 
becomes  critical. 


Using  the  transformation  matrices  involved  in  the  DH- 
method  described  above,  we  have  developed  a  new 
type  of  constraint  that  can  be  included  in  (4)  and  controls 
the  orientation  of  a  particular  part  of  the  human  model. 
The  constraint  essentially  dictates  the  direction  in  which 
a  local  axis  points  (in  terms  of  the  global  coordinate 
system)  and  can  be  used  with  any  local  coordinate 
system  (any  kinematic  link).  A  single  orientation- 
constraint  can  be  used  to  constrain  either  one  or  two 
axes  for  a  particular  local  coordinate  system.  Note  that  if 
the  directions  of  two  axes  are  specified,  the  orientation 
of  the  coordinate  system  is  dictated  uniquely.  Details 
concerning  the  development  and  use  of  this  new 
constraint  are  given  by  Marler  (2004).  An  example  is 
shown  in  Fig.  9  where  both  hands  are  touching  their 
respective  target  points  simultaneously  and  have 
specific  orientations. 


Fig.  9  Posture  prediction  for  both  hands  with  orientation 
constraints 

KINEMATIC  MOTION  PREDICTION 

While  posture  prediction  makes  it  possible  to  evaluate 
the  final  posture  of  a  virtual  human  reaching  a  target 
point,  it  is  often  important  to  evaluate  how  a  virtual 
human  moves  over  time  to  arrive  at  the  final  posture. 
Kinematic  motion  prediction  computes  the  movement  of 
a  virtual  human  by  determining  the  optimal  joint  values 
over  time  for  each  DOF.  In  particular,  kinematic  motion 
prediction  optimizes  a  set  of  joint  displacement  curves 
while  restricting  the  end-effector  to  remain  on  a  given 
path  in  Cartesian  space  (Abdel-Malek  et  al.,  2004).  This 
section  discusses  the  general  formulation  of  a  fast 
approach  and  results  in  a  21 -DOF  model.  We  have 


developed  an  optimization-based  approach  to  kinematic 
motion  prediction,  and  by  carefully  designing  the 
characteristics  of  the  B-splines  used  with  this  approach, 
we  have  achieved  near-real-time  performance. 

OPTIMIZATION  PROBLEM  FORMULATION 

The  motion-prediction  optimization  problem  involves 
finding  a  set  of  joint  displacement  curves  q(r)  that 

minimize  one  or  more  human  performance  measures. 
Conceptually,  the  design  variables  for  the  optimization 
problem  are  the  curves  that  describe  the  joint 
displacements  over  time.  Technically,  we  determine 
parameters  that  dictate  the  nature  of  these  curves.  The 
displacement  for  each  joint  is  constrained  to  lie  within 
the  joint  limits,  at  each  time  step.  In  addition,  the  end- 
effector  of  the  virtual  human  is  constrained  to  follow  a 
discretized  Cartesian  path.  The  resulting  optimization 
problem  is  formulated  as  follows: 

Find:  P'k\  k  =  l, 2, ...,bc;  z  =  l, 2, ...,n  (6) 

to  minimize:  Summation  of  human  performance 

measure(s)  evaluated  at  each  time 
step 

subject  to:  [x(q(ry))-xpa"'  (r,)]  <s;j  =  1,2,... to 

qf  <Pk  <  q'/ ;  k  =  1, 2, . . . ,  be  ;  i  =  1,2,..., n 

where  r.  is  the  time  evaluated  at  time  step  j,  ts  is  the 
number  of  time  steps,  and  be  is  the  number  of 
coefficients  used  in  the  B-spline  curves.  P'k  is  the  k*h 

control  point  for  the  /h  displacement  curve  (B-spline 
curve).  As  with  the  posture-prediction  problem,  (6)  is 
solved  numerically  using  the  software  SNOPT  (Gill  et  al., 
2002).  Components  of  this  formulation  are  explained  as 
follows. 

Design  Variables 

As  opposed  to  the  posture  prediction  problem,  the 
design  variables  with  the  motion  prediction  problem  are 
coefficients  in  B-spline  curves  that  represent  q  (r) .  B- 

spline  curves  are  used,  because  they  are  necessarily 
continuous  and  differentiable.  In  order  to  induce  smooth 
and  realistic  motion,  the  joint  trajectories  must  be  at  least 
twice  differentiable;  thus,  it  is  necessary  to  use  at  least  a 
third  degree  B-spline  curve.  Hence,  the  /h  joint  trajectory 
is  defined  as  follows: 

(7) 

where  Nk3  is  a  third  degree  basis  function  that  is 
defined  recursively.  There  are  bexn  design  variables. 

Constraints  and  Objective  Functions 

The  first  constraint  in  (6)  is  the  distance  constraint,  which 
requires  at  each  time  step  that  the  end-effector  remain  in 


contact  with  the  given  path,  which  is  determined 
analytically  by  minimizing  the  jerk  of  the  end-effector. 
The  remaining  constraints  indirectly  require  the  joint 
angles  to  remain  within  predetermined  limits.  Given 
inherent  properties  of  B-splines,  this  is  done  by 
constraining  the  B-spline  coefficients  directly. 

The  objective  functions  are  conceptually  the  same  as 
they  are  with  posture  prediction.  However,  they  are 
written  in  terms  of  B-splines  using  (7),  and  their  value  is 
summed  over  all  time  steps. 

Note  that  using  B-spline  curves  gives  a  result  with 
normalized  time  between  zero  and  one  second.  The 
resulting  motion  can  be  scaled  to  longer  durations. 

RESULTS 

Applying  the  above-described  algorithm  to  a  21 -DOF 
model  results  in  the  motion  shown  in  Figure  10.  The 
end-effector  follows  the  minimum  jerk  path  in  Cartesian 
space  with  43  discretized  points.  The  calculation  took 
only  1.09  seconds  on  a  2.6GHz  Pentium4  CPU  with 
512MB  RAM. 


Fig.  1 0  Superimposed  time  instants  of  motion  prediction 

DYNAMIC  MOTION  PREDICTION 

In  the  previous  section,  we  discuss  kinematic  motion 
prediction.  However,  when  considering  force-related 
issues  such  as  walking  stability,  fatigue,  injury,  or  energy 
consumption,  it  is  necessary  to  incorporate  dynamic 
analysis.  Thus,  in  this  section,  a  novel  and  fast 
optimization-based  inverse  dynamics  method  is 
implemented  in  order  to  predict  gait-driven  dynamic 
motion.  With  this  approach,  we  provide  a  means  of 
conducting  gait  analysis  while  considering  externally 
applied  loads,  and  the  consequent  model  is  able  to 
adapt  to  changing  loads.  By  using  gradient-based 


optimization  software,  we  are  to  determine  the  optimal 
realistic  motion  with  substantial  speed. 

Gait  motion  entails  a  typical  contact  problem  between  a 
foot  and  the  environment.  Generally,  the  contact 
problem  raises  two  issues:  1)  is  contact  between  two 
objects;  and  2)  if  there  is  contact,  what  are  the  reaction- 
forces  at  the  contact  points  and  where  does  the  contact 
occur.  With  gait  analysis,  the  contact  forces  are  ground 
reaction  forces  (GRFs),  and  the  contact  points  are  the 
points  at  which  the  foot  contacts  the  ground.  Due  to  the 
uncertainty  in  the  GRF,  it  is  difficult  to  determine  when 
the  dynamic  instability  of  falling  initiates.  With  the  current 
work,  inverse  dynamics,  combined  with  the  zero  moment 
point  (ZMP)  parameter  (Vukobratovic  and  Borovac, 
2004),  which  is  discussed  below,  successfully  addresses 
these  questions.  To  accelerate  the  optimization  process, 
the  number  and  sequence  of  foot  stepping  patterns  are 
specified,  while  other  gait  parameters  such  as  stepping 
time  and  step  length  are  kept  flexible.  A  newly  proposed 
method  employing  such  stepping  control  achieves  fast 
convergence  and  shows  motion  adaptability  to  external 
loads  in  push-and-gait. 

DYNAMIC  STABILITY  USING  THE  ZMP 

With  our  approach  to  dynamic  modeling,  we  propose 
integrating  the  idea  of  dynamic  stability  into  the 
optimization  objective  function.  Thus,  in  this  section  we 
review  the  ideas  behind  dynamic  stability. 

The  ZMP  is  an  extension  of  the  static  stability  parameter. 
As  such,  it  incorporates  gravitational  forces  as  well  as 
inertial  forces  due  to  motion.  As  long  as  the  ZMP  stays 
within  the  supporting-foot  footprint  (Fig.  11a),  then 
moment  equilibrium  between  inertial  forces  induced  by 
body  motion,  gravity  forces  acting  on  the  body,  and 
external  forces,  is  satisfied.  In  the  case  of  bi-foot  support 
(Fig.  11b),  the  ZMP  should  stay  within  the  foot  support 
region  (FSR),  which  is  a  convex  polygon  that  encloses 
all  of  the  foot-  ground  contact  points. 


Fig.  1 1  Foot  support  region  (FSR) 


Generally,  the  ZMP  falls  within  the  boundary  of  the  FSR. 
The  concept  of  dynamic  stability  comes  from  the  notion 
that  a  motion  is  more  stable  as  the  distance  between  the 
ZMP  and  the  boundary  of  the  FSR  increases.  If  the  ZMP 
is  actually  on  the  boundary  of  the  FSR,  and  if  its  position 
is  perturbed  by  external  loads  such  that  it  would  cross 
the  boundary,  which  is  physically  impossible,  then 
tipping  over  or  falling  results  unless  measures  are  taken 
to  reposition  the  FSR  (i.e.  stepping  forward  or 
backward). 

OPTIMIZATION  PROBLEM  FORMULATION 

As  does  the  formulation  for  kinematic  motion  prediction, 
the  optimization  formulation  for  dynamic  motion 
prediction  of  human  gait  incorporates  B-splines  and  is 
described  as  follows: 

Find:  0  =  {Plk,tj}  (8) 

i  =  1,2,..., be;  k  =  \,2,...,n;  j  =  \,2,...,kp 

to  minimize:  F(Q)=  jT  fDS (q(0,  t),  q(0,/),  q(0,O)  dt 

subject  to:  H] (0)  =  0;  j  =  1, 2,...,  nc 

where  0  is  a  set  of  design  variables  that  includes  the  B- 
spline  coefficients  Pi  and  knots  tj .  kp  and  nc  represent 

the  number  of  knot  points  and  the  number  of  constraints, 
respectively.  Tis  the  total  travel  time  for  a  gait  motion. 
q(0,r)  is  a  vector  of  joint-angle  profiles  and  pelvis 

translations  that  are  approximated  by  cubic  B-spline 
curves  as  follows: 

BC 

q(0, 0  =  {qk (0,  t)  |  qk  =  £ P'kNl  (t, tj);  k  =  1,2,..., n)  (9) 

i 

where  N^tJj )  are  basis  functions.  F(0)  is  an  objective 

function  that  measures  dynamic  stability.  HJ(& )  are 

equality  constraints  that  incorporate  constraint  violations 
for  each  time  step. 

The  pseudo-code  for  dynamic  motion  prediction  using 
optimization-based  inverse  dynamics  is  briefly  described 
in  Fig.  12. 

Stepping  Pattern  Control 

The  objective  function  and  the  constraints  are  implicitly 
governed  by  stepping  control.  Stepping  control  specifies 
a  set  of  foot-points  on  the  ground,  as  a  function  of  time 
and  knots,  and  is  denoted  by  T/C(r,t) : 

'Pc(M)  =  {/ 1  y^t)  =  0; ;  tJ<t<  tj ...  j :  t  =  {tj  \\<j<  n^,}  (10) 

where  y  f  (t)  is  the  y-coordinate  of  the  foot-point  /  ,  and 
t  is  a  vector  of  knots. 

Objective  Function 


Minimizing  fDS  in  (8)  increases  dynamic  stability  in  two 

ways.  First,  it  increases  the  distance  between  the  ZMP 
and  the  border  of  the  FSR.  Secondly,  it  increases  the 

span  of  the  FSR.  fDS  is  defined  as  follows: 

/DS(q,q,q,0  =  (^ZMP(q,q,q)-r(q,0)(^(q,q,q)-r(q,0) 

r(q ,0  =  min[{  Vq)  I  JP  e  'W)}]  0  1 ) 

^M(q,0  =  max[{^(q)  |  fp  e  vFc(r,t)}] 

where  xZM/>(q,q,q)  is  the  x-coordinate  of  ZMP,  and  xjp 

is  the  x-coordinate  of  the  foot-point  / .  and  are 
bounds  for  FSR.  With  two  dimensions,  these  bounds 
represent  the  minimum  and  maximum  values  for  the 
FSR,  respectively. 


Begin 

1)  Assume  initial  design  variables  including  13- 
spline  coefficients  and  knots,  for  B-spline 
approximation  of  joint  angles. 

2)  Calculate  the  joint  angles  and  their  derivatives 
from  the  design  variables 

Inverse  Dynamics  Program 

3)  Calculate  inertial  forces  induced  by  body 
motion.  Evaluate  the  objective  function, 
constraint  violations,  and  gradients  with  respect 
to  the  design  variables. 

Optimization  Program 

4)  If  the  current  point  satisfies  the  optimality 
condition,  stop. 

5)  Otherwise,  predict  new  control  parameters  for 
joint  profiles  and  go  to  step  2). 


The  joint-angle  limits  are  modeled  as  follows: 

qLk<qk(®,t)<quk ;  *  =  1,2 . «;  t  e[0,T]  (12) 

The  condition  for  moment  equilibrium  in  two-dimensions 
is  given  in  inequalities  as: 

<T”  (q>  0  -  xzmp  (q>  q>  q)  -  (q>  0  03) 

The  constraint  for  preventing  foot-ground  slippage 
ensures  that  horizontal  velocity  is  zero  for  the  foot-points 
contacting  the  ground.  The  condition  is  given  as  follows: 

Xjj,(.q,t)  =  0;fpe'¥c  (14) 

where  ijp  denotes  the  velocity  in  x-direction  for  the 
foot-point  fp . 

The  constraint  for  preventing  ground  penetration 
imposes  the  condition  that  the  foot-points  that  are  not  in 
ground  contact  stay  above  ground  level.  This  constraint 
is  stated  as  follows: 

fpe'V F(t,i)  (15) 

where  ^  (=  'Tcc )  is  a  set  of  foot-points  that  do  not 
contact  the  ground. 

The  initial/final  conditions  for  the  left  foot  are  stated  as 
follows: 

XLFT  (<T  t  =  0)  =  XIC 
XLFT  (qT  =  T)  =  XFC 

where  xLFT  is  the  x-coordinate  of  a  left  foot-toe.  The 
values  xIC  and  xFC  are  the  locations  at  the  initial/final 
time  frame. 

EXAMPLE 


Fig.  12  Pseudo-code  for  optimization-based  dynamics 
Constraints 

There  are  several  types  of  constraint  conditions  for  gait 
motion  prediction:  1)  joint-angle  limits,  2)  moment 
equilibrium,  3)  foot-ground  slippage,  4)  foot-ground 
penetration,  5)  foot-ground  slippage  condition  and  6) 
initial/final  conditions  for  the  left  foot  toe. 


In  this  example,  a  13-DOF  human  model  is  used  to 
predict  two-dimensional  push-and-gait  motion.  There  is 
an  external  force  of  250  N  pushing  backward  on  the 
shoulder.  Fig.  12  captures  body  postures  at  uniformly 
spaced  time  frames.  The  trunk  leans  forward  to  counter 
imbalance  induced  by  the  counter-clockwise  moment, 
which  is  imposed  by  the  external  force  on  the  shoulder. 
The  arms  constantly  move  to  generate  necessary  inertial 
forces,  to  keep  the  ZMP  inside  the  FSR.  The  required 
CPU  time  for  optimization  is  1 .69  sec  with  a  Xeon  3.06 


Fig.  13  Push-and-gait  motion 


GHz  Processor.  Again,  SNOPT  software  is  used  for 
optimization.  Extension  to  3-dimensional  analysis  is  on¬ 
going. 

REACH  ENVELOPE 


H  ,  = 

q 


®q !  o 

i  \'<h 


(21) 


where  qx=dq/dk,  <Dq  =  5<D/5q ,  0  is  a  (3x«)  zero 
matrix,  and  I  is  the  identity  matrix. 


The  reach  envelope  is  defined  as  all  points  the  end- 
effector  (fingertip)  can  touch  when  the  virtual  human 
moves.  Reach  envelopes  provide  an  important  feature 
for  virtual  humans,  because  if  the  virtual  human  is  to 
accomplish  a  task,  the  target  points  must  be  inside  the 
reach  envelope.  In  this  section,  we  present  an  analytical 
method  for  identifying  the  reach  envelope  and  give  an 
example  of  the  reach  envelope  for  Santos™. 


In  (1),  x(q)  represents  the  set  of  all  possible  points 

inside  and  on  the  boundary  of  the  workspace  generated 
by  the  end-effector.  At  a  specified  position  in  space, 
given  by  (x  ,y  ,z  ) ,  the  following  vector  of  constraints 

can  be  written: 


d»(q)  = 


x(q)-xp 

y(q)-yP 

z(q)-Zp 


=  o 


(17) 


Joint  limits  imposed  in  terms  of  inequality  constraints  in 
the  form  of  where  i  =  1,2, ...,«,  are 

transformed  into  equality  constraints  by  introducing 
additional  generalized  coordinates  X  =  [A,  ...  AJr  such 
that 


<7,  =  {W,  +  4!  )/2)  +  ((<?f  -  q'i  )/2)  sin  A,. ;  i  =  1, 2, ...,  n  (1 8) 


In  order  to  include  the  effect  of  joint  limits,  the  vector  of 
constraint  equations  <D(q)  is  augmented  with  the 

parameterized  inequality  constraints  in  (18).  The 
resulting  vector  of  constraints  is  given  as  follows: 

x(q)-xp 


H(q)  = 


y(M)-y„ 
z(q  )-Zp 


=  0;  i  =  (19) 


qt  -ctj  -bj  sin  A; 


where  q*  =  [qr 


XT  ]r  is  the  new  q-vector.  Note  that 


although  n  new  variables  A,  have  been  added,  n 

equations  have  also  been  added  to  the  set  of 
constraints. 


The  Jacobian  of  the  constraint  function  H(q*)  at  a 
specific  point  q*°,  is  defined  as  the  following  (3  +  «)x2« 
matrix: 

Hq.  =  (AH/  dq  (20) 

where  the  subscript  denotes  the  variable  with  respect  to 
which  the  derivative  is  taken.  The  matrix  in  (20)  can  be 
expanded  as  follows: 


The  reach  envelope  consists  of  surfaces  defined  by 
points  in  the  Cartesian  space  at  which  the  matrix  in  (21) 
is  singular.  The  detailed  procedure  for  finding  these 
singular  surfaces  is  provided  by  Yang  et  al.  (2004a).  The 
cross  section  of  the  upper  extremity  workspace  is  shown 
in  Fig.  14.  Note  that  inside  the  workspace,  there  are 
many  singular  surfaces  inside  the  boundary,  but  we  are 
interested  in  finding  the  boundary  of  the  workspace  only. 
Fig.  15  illustrates  the  workspace  boundary  for  a  95% 
Santos™  using  the  right  arm.  To  visualize  the  reach 
envelope  in  real-time,  we  have  compiled  a  reach- 
envelope  library  for  different  percentile  Santos™-models. 


Fig.  14  Cross  section  of  workspace  for  Santos’™  right 

arm 


Fig.  1 5  Reach  envelope  for  Santos’™  right  arm 


PHYSIOLOGICAL  PROCESSES 


In  addition  to  predicting  how  a  human  functions  in  terms 
of  motion,  it  is  often  necessary  to  monitor  physiological 
aspects.  Physiological  systems  for  Santos™  consist  of 
all  the  life  maintaining  systems  that  occur  within  a 
human.  The  initial  focus  has  been  on  creating  the  ability 
to  simulate  the  vital  signs  of  Santos™,  and  significant 
progress  has  been  made  with  the  ability  to  model  the 
heart  rate.  Thus,  in  this  section  we  discuss  heart  rate.  An 
overview  of  the  physiological  processes  that  effect  heart 
rate  is  given,  and  this  culminates  in  an  analytical 
expression  for  actually  calculating  heart  rate  based  on 
based  on  expended  power. 

The  human  body  is  often  compared  to  the  combustion 
engine,  and  this  analogy  provides  a  summary  of  what 
dictates  the  heart  rate.  The  muscle  fibers  act  as  pistons, 
and  adenosine  triphosphate  (ATP)  is  the  gasoline.  The 
blood,  circulated  by  the  heart,  supplies  the  oxygen 
required  by  the  ATP  to  produce  energy.  As  the  intensity 
of  activity  increases,  more  oxygen  is  required. 
Consequently,  the  amount  of  blood  supplied  to  the 
muscles  has  to  increase  in  order  to  supply  the  muscles 
with  sufficient  oxygen.  To  increase  the  blood  supply  the 
heart  increases  its  stroke  volume  per  beat  and  the 
number  of  beats  (Astrand,  1970). 

The  oxygen  consumed  by  the  muscles  in  liberating 
energy  is  called  the  oxygen  uptake  ( Vo2 ).  During 

physical  activity,  the  oxygen  uptake  increases,  and 
consequently  the  cardiac  output  also  increases  in  order 
to  keep  up  with  the  increase  in  oxygen  uptake  (Astrand, 
1970).  The  oxygen  supplied  by  the  blood  is  used 
primarily  for  the  aerobic  portion  of  the  power  generated 
by  the  muscles  and  is  based  on  the  oxygen  uptake 
response  curves,  which  are  discussed  later.  The  heart 
rate  is  directly  proportional  to  and  can  be  calculated  from 
the  oxygen  uptake  (Margaria,  1970;  Engelen  et  at. 
1996).  Consequently,  much  of  the  following  discussion 
concerns  oxygen  uptake. 

ENERGY 

Understanding  how  the  body  uses  energy  is  necessary 
for  determining  the  power  required  by  the  body,  which  in 
turn,  is  used  to  calculate  the  heart  rate.  Special  chemical 
compounds  act  as  carriers  of  energy  within  cells.  ATP  is 
one  such  substance  and  is  the  cell’s  primary  source  of 
energy  for  metabolic  processes.  ATP  breaks  down  into 
Adenosine  Diphosphate  (ADP)  with  the  release  of 
energy.  The  stockpile  of  ATP  available  is  limited  and  has 
to  be  constantly  regenerated.  Glucose,  fatty  acids,  and 
amino  acids  can  be  oxidized  to  produce  ATP  molecules. 
The  generation  of  energy  from  ATP  as  well  as  the 
regeneration  of  ATP  from  ADP,  lactic  acid,  and  other 
waste  products  also  requires  oxygen  (Astrand,  1970). 


In  addition  to  the  energy  required  by  the  muscles,  the 
body  requires  energy  for  its  functions  of  life,  like 
breathing,  digestion,  etc.  The  basal  metabolic  rate 
(BMR)  accounts  for  this  energy,  and  is  defined  as  the 
portion  of  human  energy  involved  in  homeostasis 
(merely  living  and  breathing).  This  energy  is  exclusive  of 
work  activities,  walking,  standing,  thinking,  and  digesting 
foods.  The  BMR  is  dependant  on  weight,  height,  age, 
and  sex.  It  has  units  of  kilocaleries/day  and  is  calculated 
from  the  Harris-Benedict  equations  for  men  and  women 
respectively,  as  follows  (Frankenfield  et  a/.,  2003): 

66  +  13.75w  +  5.0A  +  6.76a  (22) 

655  +9.56w  +  1.85/*-4.68a  (23) 

where  a  is  the  age  in  years,  w  is  the  weight  in  kilograms, 
and  h  is  the  height  in  centimeters. 


Although  the  actual  basal  metabolic  rate  varies  during 
the  course  of  the  day,  for  the  puprpose  of  calculating  the 
heart  rate,  it  is  assumed  that  the  BMR  is  constant.  The 
BMR  per  second  is  calculated  from  equations  (22)  and 
(23)  for  a  set  height,  weight,  age,  and  sex,  and  is  added 
to  the  power  requirement  of  the  body,  to  perform  an 
activity.  The  power  requirement  excluding  the  BMR  is 
supplied  by  the  Santos™  energy  module  (Kim  et  at., 
2004). 


STEADY  STATE  OXYGEN  UPTAKE 


The  power  to  drive  the  physical  activity  .which  is  derived 
from  the  sum  of  the  power  from  the  Santos™  energy 
module  and  the  BMR,  is  made  up  of  two  components. 
The  aerobic  component  generates  energy  utilizing 
oxygen  from  the  blood.  Alternatively,  the  anaerobic 
component  does  not  utilize  oxygen  from  the  blood.  The 
energy  supplied  during  the  first  few  seconds  of  an 
activity  is  almost  entirely  anaerobic.  (Lamb,  1978).  As 
the  duration  of  the  activity  increases,  as  a  result  of  ATP 
breakdown,  the  levels  of  Adenosine  Diphosphate  (ADP) 
in  the  muscles  increase.  This  increase  in  ADP  is  the 
spark  for  aerobic  metabolism  to  begin  (Lamb,  1978).  The 
aerobic  energy  process  utilizes  oxygen  supplied  by  the 
blood,  therby  causing  an  increase  in  the  blood  supply. 

Energy  expenditure,  including  the  BMR,  results  from  the 
breakdown  of  ATP,  and  because  ATP  is  replenished  as 
the  result  of  oxygen  utilization  in  the  mitochondria  either 
during  exercise  or  during  recovery,  there  is  a  direct 
relationship  between  oxygen  consumption  and  energy 
expenditure.  Accurate  measurements  of  the  kilocalories 
of  heat  produced  as  the  result  of  oxygen  utilization  show 
that  normal  subjects  on  a  mixed  diet  of  fat,  carbohydrate, 
and  protein,  expend  about  5Kcal  of  energy  for  each  liter 
of  oxygen  they  consume  (Lamb,  1978).  Therefore,  one 
can  obtain  the  oxygen  uptake  equivalent  of  power 
required  for  activity.  The  entire  power  consumed  by  the 


body  is  related  to  an  equivalent  oxygen  uptake  by  using 
the  following  relationship: 


Vor‘q  = 


Power  *  60 
20.92 


(24) 


where  Vo2q  is  the  equivalent  oxygen  uptake  value  of  the 
energy  required  in  L/m.  Power  has  units  of  Kilowatts, 
and  20.92  KJ  (5Kcals)  is  the  energy  equivalent  value  of 
1  L/min  oxygen  uptake.  However,  (24)  only  provides  the 
oxygen  uptake  value  once  the  body  has  reached  the 
steady  state.  Before  using  steady  state  oxygen  uptake, 
one  must  determine  the  oxygen  update  during  transient 
periods. 


start  of  phase-2  (after  time  delay  TDX )  and  is  assigned  a 
value  of  Aa  for  that  time.  Aa  is  calculated  as  follows: 

4=4(  l-e~TD'u°)  (26) 

The  third  term  of  equation  (25),  Ax( l-e+'~TD'lr,)) ,  is 
called  the  phase-2  term,  which  begins  after  a  time  delay 
of  TDX .  The  primary  components,  phase-1  and  phase-2, 

are  relatively  fast  and  are  not  affected  by  the  muscle 
mass  used  to  perform  physical  activity  (Shunsaku  et  al., 
2001).  The  amplitude  of  the  phase-2  term  is  set  to 

4=4+4  (27) 


TRANSIENT  OXYGEN  UPTAKE 

The  power  produced  in  the  transient  stages,  leading  up 
to  the  steady  state,  is  a  combination  of  aerobic  and 
anerobic  processes.  Initially  the  entire  power 
requirements  are  met  anaeorbically.  At  the  steady  state 
the  requirements  are  met  almost  entirely  from  aerobic 
processes. 

The  oxygen  uptake  of  the  body  does  not  jump  directly  to 
the  steady  state  value  but  rises  over  time  following 
exponential  curves  called  response  curves, 
corresponding  to  consumption  of  oxygen  by  the  aerobic 
processes.  The  shape  of  the  exponential  curves  and 
their  associated  time  constants  are  different  for 
increasing  physical  activity  levels  (also  called  ramp-on 
response  curve)  and  decreasing  physical  activity  levels 
(also  called  ramp-off  response  curve). 

Physical  Activity:  Ramp-on 

The  Vo2  response  to  increases  in  physical  activity  has 

been  modeled  mathematically  using  three  exponential 
terms  given  in  (25) 

Vo2(t )  =  Vo2(b)  +  4(1  - e~,lT° )  +  4(1  -  e+‘-TD'l+)  +A2(  1  - e+-m+'+)  (25) 
Each  exponential  term  represents  a  certain  phase  of 
enegetics  in  physical  activity.  Vo2(t)  is  the  oxygen 
uptake  response  in  liters/min  after  t  seconds.  Vo2(b )  is 
the  baseline  oxygen  uptake  in  liters/min.  A0 ,  Ax ,  and 
A1  are  the  asymptotic  values  for  the  exponential  terms. 
t0,  r, ,  and  r2  are  the  time  constants  in  seconds.  TDX 
and  TD2  are  the  time  delays  in  seconds,  which  are  the 

time  durations  of  time  before  phase  2  and  phase  3 
begin.  The  baseline  oxygen  uptake  value  is  the  value  of 
the  oxygen  uptake  when  the  body  is  at  rest.  This  value  is 
approximately  0.40-0.7  L/m  and  is  dependant  on  body 
mass. 

In  (25),  A0( l-4'/r°)  is  the  phase-1  term  and  starts  at 

the  onset  of  exercise.  This  phase-1  term  reflects 
circulatory  adjustments  at  the  onset  of  physical  activity 
(Barstow,  1987).  The  phase-1  term  disappears  at  the 


The  last  term,  A2(\-e +~TDl'T2)) ,  is  called  the  phase-3 
term.  This  term  becomes  significant  after  a  time  delay 
TD2 .  The  exact  cause  of  this  component  is  not  known, 
but  it  is  suspected  that  the  serial  recruitment  of  lower 
efficiency  fast-twitch  fibers  are  the  cause.  A  model  Vo2 
ramp-on  response  curve  is  shown  in  Fig.  16,  for  a 
subject  starting  from  rest  to  his/her  maximum  Vo2  value 
(Engelen  eta!.,  1996). 


Fig.  1 6  Features  of  the  three  exponential  model  used  to 
describe  oxygen  uptake  during  ramp-on 


Physical  Activity:  Ramp-off 

The  Vo2  response  to  a  reduction  in  the  level  of  physical 

activity  was  found  to  be  symmetrical  to  the  ramp-on 
model  (Engelen  et  al.,  1996)  and  is  provided  as  follows: 

Vo2(t)  =  EEVo2  +  4(1  - +  4(1  - e+‘-TD,+)  +  4(1  - e“c‘“n)  (28) 
The  only  differences  between  (25)  and  (28)  is  that 
EEVo2  signifies  the  end  level  of  oxygen  uptake  from  the 
ramp-on  activity  and  that  TD  is  a  common  time  delay  for 
both  Ax  and  A1  exponential  terms.  The  A0  term  is 
terminated  after  time  delay  TD.  The  phase-2  term  has 
amplitude  Ax ,  which  was  equal  to 

4=4+4  (29) 

The  justification  for  this  model  is  as  follows.  The  two 
main  exponential  processes  observed  during  exercise 


( Ax  and  A2  from  ramp  on  response)  are  present  at  the 

end  of  exercise.  Thus,  both  would  be  present  and 
decaying  simultaneously  during  early  recovery,  when  the 
Vo2  requirements  drop,  but  would  reduce  at  different 

rates  (Engelen  et  al.,  1996).  Fig.  17  is  a  model  ramp-off 
response  curve  (Engelen  et  al.,  1996). 


Fig.  17  Features  of  the  three  exponential  model  used  to 
represent  oxygen  uptake  ramp-off 

The  values  the  constants  involved  in  transient  oxygen 
uptake  were  taken  from  Engelen  et  al.  (1996).  The 
values  differ  for  heavy  activity,  defined  as  activity  above 
the  lactic  acidosis  threshold  (-65%  max  Vo2 ),  and  light 

and  moderate  activity  (<65%  max  Vo2 ).  In  the  case  of 
moderate  activity,  the  Vo2q  is  lower  than  the  A2 

asymptotic  value.  Therefore,  the  third  exponential  term 
drops  out,  because  no  additional  oxygen  is  required. 

Response  Curve  Implementation 

The  above-mentioned  response  curves  are  used  to 
simulate  the  changes  in  Vo2 .  If  the  Vo2  of  the  body 
needs  to  rise  to  reach  the  steady  state,  then  the  ramp-on 
curve  is  used.  If  the  Vo2  needs  to  fall  to  reach  the  steady 
state,  then  the  ramp-off  curve  is  used.  The  Vo2  of  the 
body  is  evaluated  at  every  second,  with  the  initial 
condition  being  the  baseline  Vo2 .  The  baseline  Vo2  is 

the  minimum  Vo2  that  the  body  needs  to  survive. 

Three  possible  conditions  exist  concerning  Vo2  demand: 
an  increase  in  the  power  requirement  of  the  body,  no 
change  in  the  power  requirement,  and  a  decrease  in  the 
power  requirement.  An  increase  in  the  power 
requirement  translates  to  an  increased  Vo2  demand. 
The  body  has  to  increase  its  Vo2  from  the  current  level 

to  the  new  demanded  level.  The  current  level  is  mapped 
to  the  matching  level  on  the  ramp-on  response  curve 
and  is  incremented  every  second  following  the  slope  of 
the  curve  until  it  reaches  the  required  Vo2  value,  at 


which  point  it  ceases  to  follow  the  response  curve  and 
instead  maintains  the  steady  state  until  a  further  change. 

A  decrease  in  power  requirement  translates  into  a 
decrease  in  the  Vo2  demand;  the  current  Vo2  level  maps 
to  its  value  on  the  ramp-off  curve  and  follows  the 
decaying  slope  until  it  reaches  the  required  steady  state 
Vo2  value.  The  decrease  stops  once  the  body  Vo2 

value  reaches  the  Vo2  required  at  steady  state  or  the 
baseline  values.  The  Vo2  is  not  allowed  to  fall  below  the 
baseline  value  for  the  simulation. 

STROKE  VOLUME 

The  heart  increases  the  stroke  volume  in  order  to  pump 
more  blood,  which  delivers  more  oxygen  to  keep  up  with 
the  oxygen  uptake  (Hermansen,  1970).  A  relationship 
between  the  percentage  maximum  stroke  volume  and 
the  percent  maximum  oxygen  uptake  is  given  as  follows 
(Hermansen,  1970): 

(32.553  + 14.29  l*Ln(x))*/max 
J  100  K 

where  f  is  the  stroke  volume  in  milliliter,  /max  is  the 

maximum  stroke  volume,  and  x  is  the  percentage  of 
maximum  oxygen  uptake.  The  maximum  oxygen  uptake, 
the  baseline  oxygen  uptake,  and  the  maximum  stroke 
volume  are  set  based  on  the  physical  characteristics  of 
the  virtual  human. 

HEART  RATE  CALCULATION 

The  heart  rate  is  calculated  as  follows  (Margaria,  1970): 

hr  = - ^ -  (31) 

f*Hb*(Sa-Sv ) 

where  hr  is  the  heart  rate  in  beats  per  minute,  Hb  is  the 
oxygen  carrying  capacity  of  the  hemoglobin  in  a  liter  of 
02  per  liter  of  blood,  Sa  is  the  fraction  of  oxygenated 

hemoglobin  in  the  arteries,  and  Sv  is  the  fraction  of 
oxygenated  blood  in  the  veins.  Hb ,  Sa ,  and  Sv  are 

assumed  to  be  constant  for  the  simulation  and  are  taken 
from  literature  (Margaria, 1970). 

Currently,  the  heart  rate  model  has  been  implemented, 
and  the  development  of  a  physiology-based  objective 
function  is  being  investigated.  The  underlying  principle  of 
this  objective  function  is  that  the  human  body  tends  to 
optimize  its  energy  expenditure  for  various  activities. 

SUMMARY  &  DISCUSSION 

In  this  paper,  we  have  introduced  a  new  virtual  human 
called  Santos™.  We  have  presented  optimization-based 
approaches  to  posture  prediction,  motion  prediction,  and 
dynamic  modeling.  We  have  also  provided  new  virtual- 


human  capabilities  in  terms  of  monitoring  the  body  from 
a  physiological  standpoint.  We  have  demonstrated  a 
new  approach  to  reach-envelope  analysis.  Compared  to 
currently  available  virtual  humans,  Santos™  represents 
the  next  level  in  development  and  capabilities,  and 
continues  to  grow  in  its  ability  to  aid  in  engineering 
design  and  human  study. 

A  primary  component  in  the  development  of  this  virtual 
human  has  been  the  use  of  optimization.  With  posture 
prediction  and  kinematic  motion  prediction,  using 
optimization  allows  the  avatar  to  react  to  infinitely  many 
scenarios.  The  restrictions  in  terms  of  the  variety  of 
motion  that  are  imposed  by  the  use  of  a  finite  library  of 
animations  and  the  limitations  in  terms  of  the  model 
complexity  that  imposed  by  the  use  of  inverse 
kinematics  are  avoided.  With  dynamic  analysis,  using 
optimization  allows  one  to  consider  relatively  complex 
problems  with  a  high  number  of  degrees  of  freedom, 
without  having  to  solve  equations  of  motion.  This 
approach  also  results  in  relatively  fast  dynamic  analysis. 
In  fact,  a  key  element  in  the  development  of  Santos™  is 
the  consideration  of  real  time  functionality,  which  is 
enabled  by  the  optimization  approach. 

Note  that  although  the  examples  pertaining  to  the 
different  types  of  posture  and  motion  prediction  involve 
models  with  different  numbers  of  degrees  of  freedom, 
each  project  aims  towards  a  final  version  of  Santos™ 
with  89  degrees  of  freedom.  However,  each  project  is 
currently  in  different  stages  of  development.  In  this  vein, 
the  intent  with  this  paper  has  been  to  demonstrate 
current  capabilities  and  directions  for  additional  virtual 
human  development. 

In  addition  to  the  above-mention  capabilities,  exciting 
research  is  on-going  and  concerns  further  features  for 
Santos™.  Using  a  NURBS-FEM  approach,  we  have 
developed  a  fast  method  for  analyzing  the  displacement 
of  and  stress  in  muscles.  We  are  developing  new 
approaches  for  modeling  clothing  in  terms  of  its 
mechanical  properties  and  in  terms  of  its  tendency  to 
restrict  human  motion.  A  VICON  motion  capture  system 
provides  a  means  by  which  users  can  interact  with 
Santos™  and  can  validate  predicted  postures  and 
motion.  In  addition,  validation  of  physiological  models  is 
being  considered.  An  extensive  25  degree-of-freedom 
hand  model  is  being  developed  and  will  be  incorporated 
in  the  skeletal  model  discussed  above.  The  reach- 
envelope  capabilities  are  being  extended  in  order  to 
evaluate  different  zones  in  Santos’™  environment,  in 
terms  of  potential  posture-comfort.  Finally,  vision  is 
being  incorporated  in  the  model  as  an  additional  human 
performance  measure  that  affects  posture  and  motion. 

The  emphasis  with  all  of  these  projects  is  not  just  the 
development  of  a  realistic  mannequin.  Rather,  we  are 
working  towards  a  virtual  human  that  looks,  moves,  acts, 


and  reacts  as  a  real  human  would,  not  just  in  terms  of 
outward  appearance  and  gross  movement  but  also  in 
terms  of  physiological  functions.  We  strive  to  understand 
and  model  how  and  why  humans  act  as  they  do. 
Ultimately,  Santos™  will  provide  a  design  companion 
that  essentially  works  with  an  engineer  to  shorten  design 
cycle  times  and  improve  prototype  development. 
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